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Abstract 



We consider the problem of non-existence of solutions for the following Henon-Lane-Emden system 
43 ' / - Au = 



-Av 



l N , 



when pq > 1, p, q, a, 6 > 0, and (p, q) are under the critical hyperbola, i.e. T^" + -^r^ > TV — 2. We show 
that there is no positive bounded solution in dimension TV = 3, extending a result established recently 
by Phan-Souplet in the scalar case. This solves the Henon-Lane-Emden conjecture in dimension TV = 3 
for bounded positive solutions. For the scalar cases, whether of second order (a = b and p = q) or of 
fourth order (a > = b and p > 1 = q), we show that for all dimensions TV > 3 (resp., TV > 5), there is 
no positive solution with a finite Morse index, whenever p is below the corresponding critical exponent, 
i.e 1< p < ^2±^L (reS p., l< p < Z±£2±). 
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1 Introduction and main results 

We consider the following weighted system 



j3 ■ / -Ait = \x\ a v p in il, 

-Av = \x\ b ui in Q, 



(1) 



where pq > 1 and p,q,a,b > and 51 is a subset of WL N , N > 1. 

We start by noting that in the case of the Lane-Emden equation (i.e., when p — q and a = b = 0), the 
Pohozaev inequality shows that there is no positive solution satisfying the Dirichlet boundary condition, 
whenever $7 is a bounded star-shaped domain and p > tt^§, the critical Sobolev exponent. On the other 
hand, a celebrated theorem by Gidas-Spruck [TT] states that there is no solution whenever 51 = M. N and 
p < &M| for TV > 3. This non-existence result is also optimal as shown by Gidas, Ni and Nirenberg in [TO] 
under the assumption that u = 0(\x\ 2 ~ N ), and by Caffarelli, Gidas and Spruck in [3] without the growth 
assumption. See also Chen and Li [4] for an easier proof based on the moving planes method. Also, Lin [13] 
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using moving plane methods proved similar optimal non-existence results for p < jr^ , N > 4 in the case of 
the fourth order Lane-Emden equation (i.e., when p > 1 = q and a — b — 0). 

Note that while there is no positive solution (for the Sobolev critical and super-critical exponents) in 
a bounded star-shaped domain, the non-existence result (for Sobolev sub-critical exponents) on the whole 
space is sharper as it does not assume the positivity of the solution. 

In the case of the system (UJ , one can again use the Pohozaev identity whenever SI is a bounded star- 
shaped domain in R , to establish the following non-existence result. 

Theorem A. [8l I19| Let N > 3 and let il C M. N be a star-shaped bounded domain. If 

^ + ^<N-2, (2) 

p+l q + l 

then there is no positive solution for ([T]) that satisfy the Dirichlet boundary conditions. 

By noting that the curve — Jp + +r = ^ ^ 2 is the critical hyperbola or Sobolev hyperbola, the above 
theorem states that the Liouville-type result for positive solutions on bounded star-shaped domain holds 
when (p, q) is above the critical hyperbola. It is therefore expected that - just like the case of the scalar 
Lane-Emden equation (p = q and a = b = 0) - the non-existence of solutions on the whole space M. N should 
occur exactly when (p, q) is in the complementary domain, that is when it is under the critical hyperbola. 

This is the statement of the following Henon- Lane-Emden conjecture. 

Conjecture 1. Suppose (p,q) is under the critical hyperbola, i.e., 

^ + ^>N-2. (3) 

p+l q+l V ' 

If Vl = M. N , then there is no positive solution for system {!]). 

Proving such a non-existence result seems to be challenging even for the Lane-Emden conjecture (i.e., 
when a = b = 0) for systems. The case of radial solutions was solved by Mitidieri [14 in any dimension, 
and both Mitidieri |14) and Serrin-Zou [23] constructed positive radial solutions on and above the critical 
hyperbola, i.e. —^-r + — ^-r < ~ 2 , which means that the non-existence theorem is optimal for radial solutions. 
For non-radial solutions of the Lane-Emden system, there are the results of Souto [23] , Mitidieri P3J and 
Serrin-Zou [22 who proved the non-existence of solutions in dimensions N = 1, 2, while in dimension N = 3, 
Serrin-Zou |22) gave a proof for the non-existence of polynomially bounded solutions, an assumption that 
was removed later by Polacik, Quittner and Souplet [H]. More recently, Souplet [2T] settled completely the 
conjecture in dimension N — 4, while providing in dimensions N > 5, a more restrictive new region for the 
exponents (p, q) that insures non-existence. 

Theorem B. (Souplet [21]) Assume a = b = 0. 
(i) Let N — A and p,q > 0. If {p, q) satisfies 

1 1 N-2 

> ^^> (4) 



p+l q+l N 

then system (QJ) with fl — M. N has no positive classical solutions. 
(ii) Let N > 5, and p, q > with pq > 1. If (p, q) satisfies O), along with 

p+l n q+l 



max{2— — ,2— } > N - 3, (5) 

pq - 1 pq - 1 



then system (Qp with f2 = K has no positive classical solutions. 



The Lane-Emden conjecture in dimensions N > 5 is still open. The Henon-Lane-Emden conjecture is 
even less understood. Even for the scalar case a = b and p = q ( i.e., the Henon equation), Gidas and Spruck 
in [11) solved the conjecture only for radial solutions, also showing that in this case, the non-existence result 
is optimal. For non-radial solutions, they proved some partial results such as the non-existence of positive 
solutions for a > 2 and p < jnzk (the Sobolev critical exponent for a = 0). Recently, Phan and Souplet [T7] 
showed among other things that the Henon-Lane-Emden conjecture for the scalar case holds for bounded 
positive solutions in dimension N = 3. 

Theorem C. (Phan-Souplet |17J) Let N = 3, a = b > and p = q > 1. Assume (p,q) satisfies (0), 
then there is no positive bounded solution for the Henon equation, i.e., 

-Au= \x\ a u p in R N . (6) 

For systems, Mitidieri [T3] gave a partial solution to the conjecture for radial solutions by showing the 
following. 

Theorem D. (Miditieri [14]) Let N > 3. Assume p,q > 1 satisfy 

N + minja, b\ N + minja, b} 
LJ_L -| LJ_L > n - 2, 

p+1 q+1 

then there is no positive radial solution for (QJ) in CI = M. . 

Recently, Bidaut-Veron-Giacomini [2 j used a Pohozaev type argument and a suitable change of variables 
to improve the above result by proving the following result. 

Theorem E. (Bidaut-Veron-Giacomini |2j) For N > 3, System (QJ) admits a positive radial solution 
(u,v) such that u,v € C 2 (0, oo) n C([0, oo)) if and only if(p,q) is above or on the critical hyperbola, i.e., 
when (0) holds. 

In this note, we shall prove that the Henon-Lane-Emden conjecture holds in dimension N — 3 for bounded 
positive solutions, extending the result of Phan-Souplet [T7] mentioned in Theorem C abovqj. We also give 
a few partial results for the Henon equation whether of second order or fourth order in all dimensions N > 3 
or N > 5. Here are our main results: 

Theorem 1. Suppose N = 3 and (p, q) satisfy {3|). Then, there is no positive bounded solution for (Qp with 

n = R N . 

We note that Miditieri and Pohozaev |16) have shown that the above result holds in higher dimension 
provided the following stronger condition holds: 

max{a,/3} > N - 2, 

where a := l + )p+( q + 1 anc j Q : — ( a + >i+y + > p or that they used a rescaled test-function method (as in 

pq— 1 " pq— 1 J ^ 

Lemma [TJ below) to prove the result for p, q > 1. More recently, Armstrong and Sirakov [T] proved -among 
other things- similar results for p. q > 0, by developing new maximum principle type arguments. We are 
thankful to P. Souplet for informing us of these latest developments by Armstrong and Sirakov. 

We shall also consider in the scalar case the question of existence of solutions with finite Morse index 
solutions (as opposed to bounded solutions). We get the following counterpart to the Phan-Souplet result 
in higher dimensions (N > 3). 



1 Upon receiving our preprint, P. Souplet informed us that Q.H. Phan has also proved the same result in dimension N = 3, 
as well as other interesting results in higher dimensions. Our proofs are quite similar since both are essentially refinements of 
those of P. Souplet in his groundbreaking work on the Lane-Emden conjecture for systems. 



Note that this range of dimensions falls under the corresponding critical hyperbola, i.e. N < 4 + a - 1 - : - L - 1 - 



Theorem 2. Let a > 0, p > 1 and N > 3. Then, for any Sobolev sub-critical exponent, i.e., 

N + 2 + 2a 
1<P< N-2 ' 

equation ^B\) has no positive solution with finite Morse index. 

We also have the following result for the fourth order equation, 

A 2 m= \x\ a u p in R N . (7) 

Theorem 3. Let a > 0, p > 1 and N > 5. Then, for any Sobolev sub-critical exponent, i.e., 

N + A + 2a 
1<P< N-A ' 

equation Hj) has no positive solution with finite Morse index. 

There are also various results for the cases where — 2 < a, b < and pq < 1. For that we refer to 
[2J [51 [TCI [SJ EJ HH HZ] . In [S] , similar Liouville-type theorems in the notion of stability have been proved for 
positive solutions of system ([T]) on M. N provided p > q = 1 and a = b in dimensions 

N< 8 + 3a + 8 -±^. 
p-1 

rr r'vifir'al himDrVinlci i o TV *■"' /I _L /^ _|_ 

p-i ■ 

2 Proofs 

The main tools used in our proof are Pohozaev-type identities for both systems and equations as well as 
various integral estimates. 

2.1 Proof of Theorem [j] 

The proof of Theorem Q] is heavily inspired by ideas of Souplet [21] and Serrin-Zou [22]. We use Pohozaev- 
type identities, various integral estimates, as well as some elliptic estimates on the sphere. Throughout this 
subsection, all norms refer to functions defined on the unit sphere, i.e. ||w|| m := Hulli^s^- 1 )- 

We start with the following estimate on the non-linear terms. Note that for a — b = 0, this was proved by 
Serrin and Zou [21] via ODE techniques, and by Miditieri and Pohozaev [TB] who used the following rescaled 
test functions approach for a, b > —2. For the sake of convenience of readers, we recall the proof. Interested 
readers can find more details for both scalar and system cases in |20j . 

Lemma 1. For any positive entire solution (u,v) of (QJ) and R > 1, there holds 

- r „ (b + 2)p+(a + 2) 

\x\ a vP < C R N ~ 2 —^i , (8) 

i at o (<J+2)(j+(b + 2) 

\x\ b u q < C R N ~ 2 ^^^, (9) 

IB R 

where the positive constant C does not depend on R. 

Proof: Fix the following function C.R S C 2 (R N ) with < Cfi < 1; 

1, if Id < R; 



Br, 



^^ ' 1 0, if Id > 2R; 



where ||VC_r||oo < % and ||ACr,||oo < -§?■ For fixed m > 2, we have 

\A<™M\ < r\ °> if |x| < i? or |x| > 2i?; 

|AC fl , {x)\ < C | i? -2 C m-2 j if ^ < | x | < 2R . 

For m > 2, test the first equation of ([T]) by Cr and integrate to get 

\x\ a vP(% = - f AuCS 

- - f uACS < CR- 2 f uQ~ 2 . 

JR« JB 2 r\B r 

Applying Holder's inequality we get 



\ 1/9 

M<VC£ < cr 2 (/ |x|t«') (/ \x\ b uK { R-* )q ) 



^ N \J E>2r\Br J \J B2r\Br 

< c R {N ^ q ' ] V- 2 { f \x\ b u^ ( R- 2)q ) 

\JB2r\Br J 

By a similar calculation for A: > 2, we obtain 



« N \Jb 2R \b r I 



' B2r\Br 

, J I J. 

9 

we get for pg > 1 



where - + -^ = 1. Since pg > 1, for large enough k we have 2 + - < (k — 2)p. So, we can choose m such that 
2 + - < to < (fc — 2)p which means that to < (fc — 2)p and k < (m — 2)q. By collecting the above inequalities 



|or|VCfl) < C R [(N -* q,) 7- 2]pq ( \x\ b u q ( k R ) 

< C R (N-2Kp q -l)-[(b+2) P+ (a+2)] I \ X \ a V P Q, (10) 

** B2r\Br 



and 



If \x\ h u^^T < C R [l - N ->' ] v- 2]pq l J \x\ a v"Q 
\Jw N ) \JB R 



< C R (N-2)(p q ~l)~[(a+2) q +(b+2)] / \x\ b U q C R . (11) 

J B2r\Br 



□ 



By using Holder's inequality, we can now get the following ^-estimates. 
Corollary 1. With the same assumptions as Lemma[]l we have 



L 



iN- 



(a + 2)q+(b + 2) 



v < CR" fs=f 

Br 

u CR N ~ . 



>B R 

where the positive constant C does not depend on R. 



We now recall the following fundamental estimates (see [HJ[T7]). 

Lemma 2. (Sobolev inequalities on the sphere S N ~ l ) Let N > 2, integer j > 1 and 1 < k < m < 00. For 
z £ W^iS 1 *- 1 ), we have 

II^IIl^CS"- 1 ) < C (\\ D 9 Z Wl''(S n - 1 ) + II^IIl^S"- 1 )): 

where 

' 1-± = n±I> ifk<(N-l)/j, 

m = oo, if k > (N — l)/j, 

andC = C(j,k,N) > 0. 

Lemma 3. (Elliptic LP -estimate on Br). Let 1 < k < oo and F > 0. For z e W /2,fc (^2i?.), we ftai>e 

D 2 x z\ k <clf \Az\ k + R- 2k f \z\ k ), 

B R \B R / 2 \-'B2r\Bs/4 J B 2 r\B r / 4i I 

where C = C(k,N) > 0. 

Lemma 4. (v4n interpolation inequality on Br). Let R > 0. For z S W 2,1 (B2r), we have 



J B R \B R / 2 



z\<C\R IAzI + F" 1 / |z| 

/2 \ J B 2R \B R/4i J B 2 r\B r / 4 



where C = C(N) > 0. 



By applying Lemma [I] Corollary Q] and Lemma [4j we obtain the following estimates on the derivatives 
of u and v. 

Lemma 5. We have 

BxH < C R N - 1 -i=i , 



B S 



AT-1- 



(b + 2)p + (q + 2) 



|-D*«| < C R 

Br 

where the positive constant C does not depend on R. 

For a = b = 0, the following Pohozaev identity has been obtained by Mitidieri [IS], Serrin and Zou |22) . 
It has also been used by Souplet in |21) . 

Lemma 6. (Pohozaev identity). Suppose A, 7 £ R satisfy A + 7 = iV — 2. If (u, v) is a positive solution of 
(OP, t/ien «i necessarily satisfy 



»£-*)[ l-|V« + (^-7)/ |x|V« 



F Ar+a / — + F Ar+b / — + R N f (u r Vr - R- 2 u e v e ) 
J S N-ip+l J S N-iq+l Js«-i 

+R N ~ 1 / (\u r v + ~/v r u). 



Now, we are in the position to prove Theorem [T] 
Proof of Theorem [1} Since (p, q) satisfy ([3]), then we can choose A and 7 such that ^rf > A and ^±1l > ^. 
Now, for all R > define 

From Lemma we have 

F(R)<C(G 1 (R) + G 2 (R)), (12) 

where 

Gi(fl) := i? w+Q / t; p+1 + R N+b f ^ +1 
JS"- 1 Js> 



u 

ISN-1 

and 



Ga(fl) := R N f {\D x u{R)\ + fl _1 u(U)) (\D x v(R)\ + RT^R)) . 
Js N - 1 

Step 1. Upper bounds for G\ and G 2 . Set m = 00 in Lemma [2] to get for either t = p + 1 or t = g + 1 

Nit < ||«IU < C(HZ>|«|| 1+e + |M|i) < C(R 2 \\D 2 x u\\ 1+e + HullO, 

where e > is small enough and will be chosen later. So, 

d(i?) < fl "+a+2(p+i) (||l£t,|| 1+e + i?- 2 ||«|| 1 ) 1+p 

+jR 7v +fc+ 2( 9+ i) (|| D 2 u||i+e + fl-ai^i^i+9 . (13) 

We now look for the same type bounds for G 2 . Apply Schwarz's inequality to get 

G 2 (R) < R N ([ (\D ai u{R)\ + R- 1 u(R)) 2 ) ([ (\D m v(R)\ + R^R))' 

\Js N - 1 / Vjs"- 1 

< R N (\\D xU \\ 2 + R- 1 \\u\\ 1 )(\\D x v\\ 2 + R- 1 \\v\\ 1 ). 
Then, using Lemma [2] we obtain the following upper bounds. 

\\D x u\\ 2 < Cf(||I>eD x «||i +e + ||I) x «||i)<C(ii||D»«||i +e + ||I> a! «||i) 1 

\\D x v\\ 2 < C (\\D e D x v\\ 1+e + WD.vlU) < C (R\\D 2 x v\\ 1+e + WD^Wi) . 

It follows that 

G 2 (R) < R N+2 (\\Dlu\\ 1+e + R- 1 \\D x u\\ 1 + R- 2 \\u\\ 1 )(\\Dlv\\ 1+e + R- 1 \\D x v\\ 1 +R- 2 \\v\\ 1 ).(l^ 
Step 2. The following L*-estimates hold in the annulus domain Br \ B R / 2 \ 

R IKrJUir^dr < C R N - ia+2 il + -* +2 \ (15) 

fl/2 

R WuWUr^dr < C R»- lb+2 l P X a+a \ (16) 

R/2 

R H^lli^-V < C^- 1 - a » a l (17) 

fl/2 

* HD.ull!^- 1 *- < C^- 1 - 1 ^^ 1 ^^ 1 , (18) 

R/2 

R ll^fllit^-^r < CR N - 2 - la+2 % + -? +2) +be , (19) 

fl/2 

fl ll^lliS^" 1 * < CR N - 2 - ih+ 'l*-° +2) +ae . (20) 

ft/2 



To prove ([T5)) - ([TS]) . we just apply Corollary Q] and Lemma [5j Here is for example the proof for (f2TJ)) . Apply 
Lemma [3l Corollary [T] and Lemma Q] to get 



R/2 



\Dlu\\ltlr N ^dr = 



Dlu\ l+£ dx 



R/2 



; CI \Au\ 1+t dx + C R- 2{1+e) I u 

' E>2r\Br/4 , 'S2r\Bk/4 



1+e dx 



; C 1?" I \x\ a v pil+e) dx + C R- 2{1+e) 1 

B-2r\B r /4 J B 2 r\B r /4 



< 



< C R 

The proof of (fT9|) is similar. 

Step 3 For large enough M, define following sets; 

T X (R) := {r e (R,2R); 

T 2 (R) := {r G (R,2R); 

T 3 (R) := {r g (R,2R); 

r 4 (i?) := {r £(R,2R); 

T 5 (R) := {r e (R,2R); 

T 6 (R) := {r e(R,2R); 
Using (I2TJ1) . we get 



Ar _ 2 _ (b + 2)p+(a + 2) 

pq— l 



i? 



iV- 



(6 + 2)p + (q + 2) 



-2(1 + 6) 



, ... (a + 2)q+(6 + 2) 

v(r)||i > MR 55=* }, 

. , . (6 + 2)p + (a + 2) 

u(r)\\i>MR i^ }, 
D x v\\i > MR' 1 — }, 



I> x u||i > AfiT 



(b + 2)p+(a + 2) 



}• 



(b+2)p+(a + 2) 



2./,' 



Dlv\\\+l> MR- 2 - 
D 2 x u\\\+\>MR- 2 



}2„,||l+e„JV-l, 



(a + 2) g +(h+2) 



"'}, 



+bci 



(6 + 2)p + (a + 2) 



+ae 



}■ 



ID^II^r— A dr 



_ w+2+ (^ t 2)p ± (a ± 2 1 _ o 



(fr + 2)p + (a + 2) 



R -^ ;- ■ p - q _i ■ ^lYeiR^RK^MR- 2 - %«-i +ae = Mlre^lii" 1 . 

Therefore, choosing large enough M, we get |F 6 (i?)| < R/7. Similarly, using (flo")) - ([!§)) ■ one can see |rj(i?)| < 
R/7 for 1 < i < 5. Hence, for each R > 1, we can find 



Re(R,2R)\\jT t (R)^ 



(21) 



We now have the following upper bounds on ([15)) and (|14p for the radius R given by (|2"TT) ; 



where 



Gi(fl) < C J^-HH-aip+l) /#(, — - + "< It-t + /,. 



^+a + 2(p+l) /fl(- ( ° + ^l ( 1 t+2) -2+60l^ 
_ c #V+6+2( g+ l) /^(-iil^^i-2+a,)^ 



(q + 2)q+(b + 2) \ P+l 



i?" 



PQ— 1 

pq-1 



< C (R- ai{e) + R- a 'i {e) ) , 

(a + 2)q+(b + 2) 



oi(e) = (p + l) 
oi(c) = (« + !) 



pc?-l 

(fr + 2)p+(a + 2) 

z H ae 

P9- 1 



, 1 iV + a 

- be 2 - 



1 + e 
1 



-2- 



p+1 

iV + 6 

q + i 



Also, 

g 2 (r) < cR N ^(R(- ib+ r+r 2) -^)Th+R 
(fl(- ( ° +2 ^r 2) -^)^ +J ^- 2 - 



(fe + 2)p + ( a + 2) | „^ 1 ~ 2 (b + 2)p + (a + 2) 

pq-1 



(g + 2)q + (b+2) ' 
pq-1 



< C i?- Q2(e) , 



M », 9JL 1 A , , M , (b + 2)(p+l) + (a + 2)(q + l) 

a 2 (e) = -N - 2 + 4 - (a + b)e 



where 

(hffl = -AT-2 4- 

1 + e V P? - 1 

Hence, from (fT2|) we get 

^(■B) < C (Gi(i?) + G 2 (R)) < C R-^, 

where r] e := min{ai(e), a'^{e), 0.2(e)} and the positive constant C does not depend on R. By a straightforward 
calculation, we have 

a2 (0) = - iV + 2 + (6 + 2)(P+1) + (a + 2)(9 + 1) >0 iff ^ + ^>A-2. 

pq — 1 p + 1 o+l 

Also, 

rtrt » n iff (« + 2)o+(o + 2) A + o. 

ai(0)>0, iff > — — -, (22) 

pq - 1 p + 1 

oU0)>0, iff ^M^>^. (23) 

1W pg-1 9 + 1 

Now, if p and g satisfy (j3|), then ([50)) and (J5TJ) hold, and we can therefore choose r\ t > for small enough 
e > 0. We now conclude by sending R — > 00 and get the contradiction. 

□ 

2.2 Proof of Theorem |2j 

We recall that a critical point u £ C 2 ($7) of the energy functional 

/(«):= /i|V«| 2 --^|sr«* +1 . 
Ja 2 p + 1 

is said to be 

• a stable solution of ([6]) if for any <\> 6 C* (f2), we have 



/„„(<£):= / |W|'-p / |x| a U P-V>0. 
Jn Jn 

• a stable solution outside a compact set E C £7 if I U u{4>) > for all </> £ C^(Q \ E), also u has a Morse 
index equal to to > 1 if to is the maximal dimension of a subspace A m of C^(il) such that I U u{4>) < 
for all e A m \ {0}. 

• a solution with Morse index to if there exist 0i, ..., (f) m such that X m = Span{tfii, ..., m } C C^(Q,) and 
J u „(0)<Oforall0eX m \{O}. 

Note that if u is of Morse index to, then for all € C*($7\E) we have I uu (4>) > 0, where E = U™ 1 supp(<fi i ), 
and therefore w is stable outside the compact set S C O. 

We shall need the following lemma. 



Lemma 7. Let CI C M. N and let u G C 2 {Cl) be a positive stable solution of (0|). Set f(x) = \x\ a , a > 0, then, 
for any 1 < t < — 1 + 2p + 2y / 'p(p — 1) we have 



t+i . , ,,i+£ 



flVulV" 1 + /(ar)u t+p ) </> 2m < C / /(x)~— |V0| 2 5^r, (24) 



/or aZZ ^ G C*(f2) wra£/i < < 1 and /or Zarge enough m. The constant C does not depend on CI and u. 

Proof: The following proof also holds true for weak solutions. The ideas are adapted from [51 [HI EJ. Note 
first that for any stable solution of © and r\ G C^(Cl), we have the following: 

p [ \x\ a vTW < [ \Vn\\ (25) 

Jn Jn 



x\ a u p n = / Vu-V?7. (26) 

n Jn 

Test (|26)) on 77 = u*</> 2 for G C^(O) for an appropriate t G R that will be chosen later, to get 

>> Jn 

= t / |Vu|V~V + 2 / u f Va 



Apply Young's inequalitjo to ( |Vu|u 2 0) (u 2 |V</>| ) to obtain 

(t - e) / |Vu|V~V <C e f M t+1 |V0| 2 + / |a:|V +J > 2 - 
Jn Jn Jn 

Now, test (|25|) on u~^~<j) to get 

p / |*|V+V < ^i^ f |v«|V-V + / « t+1 |v0| 2 

Jn 4 J n 7n 



(27) 



n 
+ (i + 1) / m'Vu 

(*+l) 



< 



+ 2e) f |V M |V~ V + (C' e t + C'U) f u t+1 \V<j>\\ 
J Jn Jn 



where again we have used Young's inequality in the last estimate. Combine now this inequality with (|27[) to 
see 



^ ± ^-2 £ \ r . .. ,,_.„ /^ii! + 26 



p- 



4 



x|V+V < — * ! C e + C^ t + C' t / u f+1 |V</)| 2 . (28) 



* - e I Jn \ t-£ 



For an appropriate choice of t, given in the assumption, we see that the coefficient in L.H.S. is positive for 
e small enough. Therefore, replacing <f> with c/) m for large enough m and applying Holder's inequality with 
exponents |iE an d i±£ we obtain 

Note that both exponents are greater than 1 for t given in (i) and (ii). 



2 For any a,b,e > 0, ab < ea? + C(e)b 2 , for some C(e). 
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On the other hand, combining ([27)1 and ([25)) gives us 

f |V U |V-V<£e, t /u t+1 |V</)| 2 . 

Similarly, replace <fr by 4> m and apply Holder's inequality with exponents |£y and ^y to get 

/ iv.iv-v 2m < D>: >t>m f ixr^iv^i 2 ^. 

This inequality and dM} finish the proof of (J24J) . 

Now, we are in the position to prove the theorem. 
Proof of Theorem [2} We proceed in the following steps. 



Step 1: We have the following standard Pohozaev type identity on any ft c 



piV 



^±^ / \x\ a u p+1 - ^^ / |Vu| 2 = — !— f \x\ a u p+1 x-v+ f x-Vuv-Vu-- [ |Vu| 2 ;e-za (30) 
P+Usi 2 J n P+lJdn J an 2 J an 

To get (f3"0")) , just multiply both sides of ^ by x ■ Vu, do integration by parts and collect terms. 
Step 2: The following estimates hold: 

|Vu| e L 2 (R N ), 
\x\ a u p+1 £ L 1 ^). 

First recall that u is stable outside a compact set S C ft. To prove our claim, we use (|2"3| with the 
following test function £r 6 C^(R N \ S) for i? > i? + 3 and S C Br ; 

f 0, if |x| < i? + i; 
e«(a;):= < 1, if i?o + 2< |x| < i?; 
[ 0, if \x\ > 2R; 

which satisfies < £ R < 1, ||V6?.||l~(b 2J? \-B„) < g and ||V£ fl |U-(i3R 0+2 VB„ 0+1 ) < C flf) . Therefore, 

|V«| V- 1 + \x\ a u t+p ) <C Ro +C R"-^ 1 -^, 



R +2<\x\<R 



for all 1 < t < -1 + 2p + 2^(^-1). 

Now, set t = 1 and send i? — >• oo. Since TV < Ip+°+ ^ , we see J RN |Vw| 2 < oo and J KJV |x| a u p+1 < oo. 

Step 3: The following equality holds 

|Vu| 2 = / \x\ a u p+1 . (31) 



Multiply © with u( R for C, R £ C*{R N ) which satisfies < £ R < 1, ||VCj?||oo < § and 

1, if Id < H; 



Then, integrate over I?2,r to get 



^^ •"" I 0, if \x\ > 2R 



\x\ a u p+1 ( R - \Vu\ 2 Cr= V( R -Vuu. (32) 

E>2R J E>2R J B2R 
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By Holder's inequality, we have the following upper bound for R.H.S. of (|3"2"j) . 
\ VCr-Vum] < R- 1 / \\7u\(\x\^u) \x\~^ 

J B2R J B2R 



P-1 



R-'l I Nu\ 2 ) I I IrfV+M" ( / \ x \-& 1 ""'" ' 



-B2.R / \J E>2R / \J B2R 

= Jjw-w-'ff |Vu| 2 V ( f \x\ a u p+1 Y +1 . 
Therefore, from Step 2, there exists a positive constant C independent of R such that 

_ Af(p-l)-2(q + p+l) 

VCfl -Vuu\ < C R ^p+d 

B 2 R 

Since N < 2 (P+_ a + 1 ) ) we have lim^^ | / VCr • Vw u| = 0. Hence (g2J implies (|5Tj) . 

Step 4: we have 

p + 1 2 J E ™ 

Apply Lemma for £ = 1 with the following test function 4>R <E C\ (R N \ E) for i? > 2i? ; 

f 0, if |x| < R/2; 
4> R (x) := < 1, if i? < |z| < 2R; 
[ 0, if |x| > 3i?; 

which satisfies < 4> R < 1, ||V0 fl || L ^ (B3R \ BR/2) < ^ to get 



/ (|Vu| 2 + M<V +1 ) < Ci?^- 21 ^ 11 . (33) 

•* B2r\Br 



Now, define the following sets for large enough M; 

Oi(R) := {r G(R,2R); ||I> x u(r)||l > ATR"* 1 ^}, 

2 (i?) := {r £^,2^); ||u(r)||^>Mi?- 2i ^ ii - a }. 

From (J33)) . we have 

Jr 

> i?- Ar+ ^^ i+a |0 2 ( J R)|i? Ar - 1 Afi?-^^ 1 - Q = M\e 2 {R)\R-\ 

Similarly, one can show |#i(-R)| < R/M. By choosing M large enough we conclude \0i(R)\ < R/3 for i = 1,2. 
Therefore, for each R > 1, we can find 

4=2 

£e(.R,212)\(jAi(fl)^0. 

2 = 1 

Now, apply Pohozaev identity, (|30]) . with f2 = 5^. to see that R.H.S. converges to zero if R — ► 00 for 
subcritical p, i.e. A < lP+ a + J Hence, 

N-2 f a AT + a /• . .„ p+1 

P + 1 
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From this and (f3~Tj) , we finish the proof of Step 4. 

□ 
Remark: For the Sobolev critical casep = NJ ^^ 2a , using the change of variable w := u(r 1+ f ) and applying 
well-known classifying-type results mentioned in the introduction for the Lane-Emden equation, one can see 
all radial solutions of (j6]) are of the following form 

u e (r) := k(e)(e + r 2+a )^ , (34) 

N-2 

where k(e) = (e(N + a)(N — 2)) 2 < 2 + a > . Then, from the classical Hardy's inequality it is straightforward to 
see u e is stable outside a compact set Br , for an appropriate Rq. Note that for — 2 < a < 0, by Schwarz 
symmetrization (or rearrangement), it is shown in [9] that all radial solutions of ([6]) with p = — ^ jt a and 



N > 2 are of the form (|34j) . 

2.3 Proof of Theorem 02 

We recall that a critical point u e C 4 (J7) of the energy functional 

/(«):= /i|Au| 2 --Uz|<V'+ 1 , 
Vn 2 V + 1 

is said to be a stable solution of ([7]), if for any <f> € C 4 (f2), we have 

/„„(<£) := / |A0| 2 -p / |x| q U p-V 2 > 0. 
in Jn 

Similarly to the second order case, one can define the notion of stability outside a compact set, which contains 
the notion of solutions with finite Morse index. 
We first prove the following estimate. 

Lemma 8. Let fl C R. N and let u £ C 4 (f2) be a positive stable solution of f?|). Then, for large enough m, 
we have for all 4> £ C 4 (£l) with < <f> < 1, 

(\Au\ 2 + \x\ a u p+1 ) if> 2m < C f \x\~^ a \T{^)\^, (35) 

n Jn 

where T((f)) :— \A<fi\ 2 + |V0| 4 + |A|V0| 2 | + \V(f> ■ VA</>|. The constant C does not depend on fl and u. 

Proof: For any stable solution of ([7]) and r\ 6 C 4 (f2), we have the followings: 

p f M Q u p -y < / |Ar,| 2 , (36) 

Jn Jn 



i a 
Test dSTJ on r] = ucj) 2 for e C 4 (fi) to get 



f \x\ a u p n = f AuAn. (37) 

Jn Jn 



x u' 



n 



a vP +1 4> 2 = Au A (u(f> 2 ) (38) 



Also, test (f36|) on u(f> and use (j38|) to get 



(p- 1) / |x| a u p+ V < / |A(u0)| 2 - f \x\ a vP +1 4>- 
Jn Jn Jn 



V p+U2 
i> Jn 

A{u<j>)\ 2 - f AuA{ucj) 2 ). 
a Jn 
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By a straightforward calculation, one can see that the following identity holds: 

\A(u(j))\ 2 - AuA(u(j) 2 ) = 4|Vu • V0| 2 + u 2 |A0| 2 - 2uAu\V(/>\ 2 + 2Vu 2 • V0A0. (39) 

Therefore, we have 

(p-1) f \x\ a u p+1 2 < 4 f |Vu| 2 |V0| 2 + [u 2 \A0\ 2 -2 f uAu\V<j>\ 2 
Jn Jn Jn Jo 

+2 / Vw 2 • V0A0. 

A simple integration by parts yields 

f \Vu\ 2 \Vct)\ 2 = [ u{-Au)\V<j)\ 2 + \ f w 2 A|V0| 2 , (40) 

Jn Jn 2 J n 

which then simplifies the previous inequality to become 

(p-1) [ \x\ a u p+1 <f> 2 < 6 / u(-Au)\V(t)\ 2 + [ u 2 (-\A<t>\ 2 + 2A|V0| 2 - 2V0 ■ VA0). 
jsi Jn Jn 

Therefore, 

/ |x| a u p+ V 2 < C f u|Au||V0| 2 + f u 2 L((j)), 
Jn Jn Jn 

where L(<j>) := \A<j>\ 2 + 2|A|V0| 2 | + 2\V<j> • VA<£|. 

On the other hand, from (l39l and (l40l). one can see 



(41) 



/ |A(w0)| 2 = / AuA(u0 2 ) +4 / |Vu • V0| 2 + / u 2 |A0| 2 - 2uAw|V0| 2 - 2 / M 2 cfo(V0A0) 
jn Jn Jn Jn Jn 

= f \x\ a u p+l <i> 2 + 6 / u(-Au)\\7cf)\ 2 + f u 2 (-\Acj)\ 2 + 2A|V^| 2 - 2V0 • VA^) . 
Jn Jn Jn 

By combining (|4Tj) , the identity A(u<f>) — <f>Au + 2Vu-V4> + uA(f> and Young's inequality, we get the following 
estimate 

f \Au\ 2 cj) 2 < C [ u\Au\\V(t)\ 2 + C [ u 2 L((t)). 
Jn Jn Jn 

(\x\ a u p+1 + \Au\ 2 ) </> 2 <C f u\Au\\V(f>\ 2 + C f u 2 L{0). 
in Jn Jn 

Now, replacing <j) with (f> m for large enough m > and applying Young's inequality we end up with 
(\x\ a u p+1 + \Au\ 2 ) <f> 2m < C [ ulAuHV^I 2 ^" 1 - 1 ) +C f u 2 L{(f> m ) 



Therefore, 



< e / \Auf(/> 2m + C e / u 2 |V0| 4 2(m - 2) +C / u 2 L{cj) m ). 
Jn Jn Jn 

Then, for large enough m 

' (\x\ a u p+1 + \Au\ 2 ) 2m < C f u 2 2(m ~ 2) T(0), (42) 

Jn 
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where T(<p) := \A(f>\ 2 + |V0| 4 + |A|V0| 2 | + \V</> ■ VA<j>\. Now, apply Holder's inequality to get 

[ «V (m " 2) r(^) = [ \x\^u 2 ^ m -^\x\-^T(<f>) 

Jn in 

Choosing m large enough, say 2(m — 2) £ rp- > 2m, from (|42|) we finally get the desired inequality 

f (\x\ a u p+1 + \Au\ 2 )4> 2m < C f \x\~&T&(<f>). 
Jn Jn 

Proof of Theorem [3j We proceed in the following steps. 



□ 



];iV 



(43) 



Step 1: We have the following standard Pohozaev type identity on any ft C 

E±l f N •„*■! _ *Ll± [ |Au| 2 = 4t/ M^+W-J/ |A«|Vi, 
P+l in 2 ,/ n p + 1 ,/ an 2 y an 

- / VAm ■ i/x • Vu + / AuV(x • Vu) • i/. 
■/an Jan 

To get (|43p. just multiply both sides of (J7J by x • Vu, do integration by parts and collect terms. 

Step 2: we have 

|Au| e L 2 (R N ), 
\x\ a u p+1 e L\R N ). 

Since u is stable outside a compact set S C fi, using (1331) with the following test function £r G C t ^(M jv \S) 
for R > R Q + 3 and S C S flo ; 

r 0, if |x| <i? + i; 

£ R (x):= { 1, ifi?o + 2< |x|<i?; 
[ 0, if |x| > 2.R; 

which satisfies < £ R < 1, H-D^rIU-'CB^VBb) < % and ||-D 1 6?||l~(b Ro+2 vb„ 0+1 ) < C 1 ^ for i = 1, • • ■ ,4, 
we get 

4(p+l) 2 



'R +2<|x|<-R 

p 
Step 3: The following equality holds 



/ (|Au| 2 + |x| a U P +1 ) < C Ro +6 R? 

J Rn+2<\x\<R 



For subcritical exponents, N < P p-\~ > we see Ji« Au| 2 < oo and J RN x| a w p+1 < oo. 



x|V +1 = / |Au| 2 . (44) 

R N Jr n 

Multiply © with u£r for ( R € C*(B 2R ) which satisfies < Cr < 1, ||-D'C«lloo < % for i = 1, • • • ,4 and 

1, if x < i?; 



Cb(«) — i 0; if j^i > 2R. 



Then, integrate over B 2R to get 

f \x\ a uP+ l ( R - f \Au\ 2 ( R = f uAuAC R + 2 f AuVu-V( R =:I 1 (R) + I 2 (R). (45) 

J B2R J E>2R J B2R J Din 
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By Holder's inequality, we have the following upper bound for ii(i?), 
\h(R)\ < R~ 2 J \Au\{\x\^u) \x\~ifc 



R- 2 [ I \Au\ 2 \ [/ \x\ a uP +l Y +1 ( I \x\~fr """ 



B2R / V* i?2.R / V*' B2R 

1 1 

= iJ^w-pfi- ( / lAul 2 ) 2 [ / |x| a u p+lX " 



p-i 



' Bin / \J B2R 

Therefore, from Step 2, there exists a positive constant C independent of R such that 



JV(p-l)-2(q + p + l) 

|ii(-R)| < CR ^^ . 



2(p+q+l) 



Since N < l _ ^ , we have lim^^oo |Ji(i?)| = 0. Now, we consider the second term in R.H.S. of (|45l) . 
Apply Young's inequality for a given e > (we choose it later) to get 

\h(R)\<ef \Au\ 2 + C e f |V M | 2 |VCa| 2 , 

JR N Jb 2 r 

Using Green's theorem we get 

f \Vu\ 2 \VCr\ 2 = I u(-Au)\VCb\ 2 + \ I u 2 A\VCR\ 2 =:h(R)+h(R)- 
•* B2R J B2R J B2R 

By the same discussion as given for I\{R) one can see liniR^oo \I^(R)\ = 0. For the term Ia(R), we apply 
Holder's inequality again 

|I 4 | < iT 4 / \x\^u 2 \:r — 



2R 



< r- 4 { I \x\ a uP +1 ) P+1 ( f y-Fi V; ' 



B2R / \J B_ 



■2tf 



N(p-l) 2a A I C , i\ P+1 

i?-(FFTr-FFT- 4 / \ X \ a vP +l 



2/? 



By Step 2 and sending i? to infinity we get, lim^^oo \I±{R)\ — 0. Since lim^^oo \I 2 {R)\ < e/ K w |Au| 2 for 
any e > 0, we have hin^^oo |/2(-R)| = 0. Therefore, (|4"4")l follows. 

Step 4: The following equality holds 

p + i 2 ; V M 

Apply Lemma H] with the following test function 0# G C^R^ \ S) for i? > 2i? ; 

f 0, if |x| < 72/2; 

<f> R (x) := < 1, if i?< |z| < 2i?; 
[ 0, if |x| > 3i?; 

where < (f> R < 1, ||£ , >fl||L~(s 3fi \s ii/2 ) < %■ Then, we get 

,-, ,1 jvr 2(2p + 2 + a) 

|Au| 2 + |x| a w p+1 < CR N p=i— . (46) 

B2r\Br 
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On the other hand, we arc interested in similar upper bounds for the following terms 
Ji(JZ):= J |A«||V«| and J 2 (R) := f \Au\\D 2 x v 

J B2r\Br J B2r\Bji 

For the first term, Ji(i2), using Schwarz's inequality we have 



|A«||Vtt| < ( f \Au\A ( J \Vu\ 2 \ 

B2r\Br \J B2r\Br J \J B2r\Br J 



From standard elliptic interpolation estimates, L 2 -norm version of Lemma [4j we have 



B2r\Br 



IVwl 2 ■ CR 2 I \Au\ 2 + CR- 2 I 



Bai\B R /2 



B&n\Bn/ 



~,N- 



W(p-l) 2a 



< CR" p-i +^ + i?-fFFTr-FFT 



\x\ a u p+1 



CR ~ 



v-1 I AT 2(2p + 2 + o 



N 



1-2+2 



R~ 



t(n- 2 -^^) 



\x\ a u p+1 



Since / RJV \x\ a U P +1 < oo and iV < 2(2p + 2 + a) , for i? > 1 we have 



,„ P-l ( AT 2(2p+2 + a) „\,, 

|Vw| 2 < CiJ^+rl^ p-^r— +2J+2 

1 B 2 r\B r 

Therefore, 

/" , . ,, , E-flV 2(2p+2 + a) N 

/ |Au||Vu| <CR7+*( N p^r— J 

Similarly for the second term, J 2 {R), using Lemma [3l i.e., 



+ i 



(47) 



I \D 2 x u\ 2 <C[f \Au\ 2 + R- 4 

J B 2 r\B R \J B4r\B R / 2 J B4r\B R / 2 



and similar type discussions one can see 



B2r\Br 

Now, define the following sets for large enough M 



f \Au\\Dlu\<CRM"- 2 -m^). 

J Bo n\Bn 



= {r e (R,2R); ||A, U (r)|| 2 > Mr 2 '^"'}, 



= {r e(R,2R); \\u(r)\\ p p H > MR 



p-i 

2(2p+2 + q 



'}• 



MR) 

A 2 (R) 

A 3 (i?) 
Ai(R) 

In the following, we shall find a bound for the measure of the above sets. From (|46|) . we have 

c2R 



p / JV , 2(2p + 2 + a) \ . n 

{r e(R,2R); \\A x u(r)\V x u(r)\\\i > MR~^^ + p-i J+ 1 }, 

„ p / JV i 2(2p + 2 + a) \ 

{r e(R,2R); \\A x u(r)Dlu(r)\\i > Afi? _ FFrlir+ p-i J}. 



7\r_l_ ^ ^P + ^i-iJ i „ 



/? 



IKrJUgir"- 1 *- 



> .R-^+^-t 



2(2p + 2 + q) .. n 2(2p+2 + q) 

+a |A 2 (fl)|i? JV - 1 Mfl s=i Q = M|A 2 (i?)|i?" 



(48) 
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Also, from (g7} 

r2R 



C > R-fTi(~ N + 2{2 l + -i +a) )- 1 f HA^u^lV^WIIIir^- 1 * 

Jr 

p ( AT . 2(2p+2 + a) \ , *r i P { N i 2(2p + 2 + a) \ . .. 

> R^tt(- n+ p-i )- 1 |A 3 (i?)|J 1 ' jV - 1 Af^?TT(T+ P-i ' ) +1 = M |A 3 (i?)|iT 



Similarly, from (|4"8")l and (|46l) we get |Ai(i?)|, |A4(i?)| < R/M. By choosing M large enough we conclude 
Aj(-R)| < -R/5 for i = 1, • • • ,4. Therefore, for each _R > 1, we can find 



i=4 



^6(i?,2i?)\|jA,;(i?)^0. (49) 

Then, from the definition of R and A^ for i = 1, • • • ,4, we have 

/ |A B u(.R)||l£u(.R)| < OR**^ 



-■ -.--■ + ■ ■ 
p-i 



(50) 



'\x\=R 

Using (1431) with £7 = _B 2i =, \ i3^, one can see 



VAu • vx ■ Vu 
\=R 



\A x u(R)\\V x u(R)\ < CR^( N - 2J ^^ 1 ') (51) 

x\=R 

f \A x u(R)\ 2 < CR^- 2 - 122 ^ 1 )-' 

J\x\=R 

f u* +1 (R) < Ci?*( JV - H2 f t ^ i )- a - 1 (53) 

J\x\=R 



(52) 



- p / » r 2(2p + 2 + n) \ 

<Ci?FFT( Af p^T" ). (54) 



Now, applying the Pohozaev identity, (l43"|) , with O = i?^, and using (f50"|) - (j54")) . R.H.S. of (|43j) . converges 
to zero if R — > oo for subcritical p, i.e. A < _L-P±_±£i. Hence, 

JV-4 f ,. |2 JV + a / , |0 „ +1 
\Aur = / \x\ u p+ . 



From this and (|44l) . we finish the proof of Step 4. □ 
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